Domain adaptation (or transfer learning in a more general case) refers the problem of learning a discriminative model in the presence of the shift between training (source) and test (target) distributions. Multiple shallow transfer learning methods bridge the source and target domains by learning invariant feature representations or estimating the instance importance without using target labels. Recent deep transfer learning methods leverage deep networks to learn more transferable feature representations. They embed domain adaptation in the pipeline of deep learning, which can simultaneously disentangle the explanatory factors of variations behind data and match the marginal distributions across domains.
In domain adaptation, the most challenging is the unsupervised case when labeled instances are not available in the target domain and thus cannot help in reducing the domain shift. This challenge becomes critical when the joint distributions P(X,Y) of features and labels change from one domain to another, which is a natural scenario in practical applications. A recent work gives a new approach to correcting the domain shift based on the theory of kernel embedding of marginal distributions. In the unsupervised case, when target labels are unavailable, the adaptation is executed by minimizing the discrepancy between marginal distributions.
Maximum mean discrepancy (MMD) criterion, derived from a two sample test, is the main element of learning deep models for domain adaptation. It characterizes a joint distribution of input data and their labels via kernel mean embedding.